tr null
Conditional independence testing under misspecified inductive biases
Then, we study the performance of regression-based CI tests under misspecified inductive biases.
Transformers learn to implement preconditioned gradient descent for in-context learning
Several recent works demonstrate that transformers can implement algorithms like gradient descent. By a careful construction of weights, these works show that multiple layers of transformers are expressive enough to simulate iterations of gradient descent.
